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A Study exaained the validity of using the Hinnesota 
Job Description Questionnaire to aeasure occupational reinforcer 
patterns (OBPs). Although OPP profiles (twenty-one adjusted scalesi 
were developed a decade aao to operationalize a key variable in the 
Theory of Hork Adjustment aofquist and Davis, 1969), little validity 
evid^ce exists. An exaaination of the process by which adjusted ORP 
scores are created froa the ipsetive unadjusted scores suggested 
several threats to validity. In addition, correlaticns of the OHP 
scales with Census Bureau and Department of Labor Data on pay, 
education, occupational self-direction* and seventeen worker trait 
requireaents show that (1) aaong the seven adjusted OBP scales for 
which there was good validation criteria, five were of questionable 
validity, (2) aaong the re»ainin<J fourteen adiusted reinforcers, 
several do not correlate in intelligible ways with the job 
characteristics data, and (3\ the adjusted scores are no sore valid 
than are the unadjusted scores. Discriminant analyses showed that the 
two sets of scales dlscrininate equally well aaong cccupations, but 
different reinforcers are sost iaportant for separating the groups. 
The conclusion froa the discriminant analyses is that whereas the 
unadjusted scales provide limited but useful inforaation, the 
adjusted scores provide no new infornation and say be less 
inter pretable than the unadjusted scores. Soae specific recoaaended 
uses of the OBP scores should be exaained. (Possible explanations for 
the failure of the scaling aethod to produce valid absolute OBP 
scores are discussed.) (Author/PV) 
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Introductory Statement 

The Center for Social Organization of Schools has 'wo primary 
ob.iectives: to develop a scientific knowledge of how schools affect 
their students, and to use this knowledge to develop better school 
practices and organization. 

The Center works through four programs to achieve Its objectives. 
The Studies in Sch ool Desegrega tion program applies the basic theories 
of social organization of schools to study the internal conditions of * 
(.'esegregated schools, the feasibility of alternative desegregation 
policieH, and the interrelation of school desegregation with other equity 
issues such r,s housing and job desegregation. The School arganiza tion 
program is currently concerneu with authority-control structures, task 
structures, reward syscoms, and peer group processt^s in schools. It 
has produc-d a large-scale study of the effects of open schools, has 
developed Student Team Learning Instructional processes for teaching 
various subjects in elementary and secondary schools, and has produced a 
computerised svstera for school-wide attendance monitoring. The Scliool 
Processes and Career I)cjvejj5jp_me_nt program is studying transitions from 
higli school to post secondary institutions and the role of schooling in 
the development of career plans and the actu- lization of labor market 
outcomes. The Studies^ in Delijujutnic;' £nd_Sc_}^^^^ program is 

examining the interaction of school environments, school experiences, and 
individual charai:tertstics in relation to in-school and later-life 
de I ;.nquencv . 

This report, prepared bv the School Processes and Career Development 
program, examines the validity of using the Minnesota Job Description 
(Questionnaire to measure occupational reinforcer patterns. 
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Ahs tract 

Although occupational refnforcer pattern' (ORp) profiles were developed a decade 
aj'.o to t perat lonalize a kev variable in the Theory of Work Ad justment (Luf- 
qulst & Dawis, 1969). little evidence exists to support the validity of those 
scores. This study tests the assumption that the 21 adjusted ORP scales are 
valid for their Intended purpose — to measure absolute levels of reinforcers in 
different occupations. Specifically, an examination of the process by which 
adjusted ORP scores are created from the Ipsative unad .lusted scores suggests 
several threats to validity. In addition, correlations of the ORP scales with 
data troni the Census Bureau and the Department of Labor on pav, education, 
occupational self-direction, and 17 worker trait requir.-ments show that: (a) 
amonR the 7 adjusted ORP scales for which there are good validation criteria, 
one reinforcer seems valid, one is clearly not valid, and the other five are 
of questionable validity, (b) anong the remaining lA adjusted reinforcers, 
several do not correlate in intelligible wavs with the job characteristics 
data, and (c) the adjusted scores are no more valid than are the unadjusted 
scores— a finding that contradicts the developers' statements about the rela- 
tive validity of the two sets of scores. Discriminant annlvses with the 
adjusted and unadjusted scores show that the two sets of scales discriminate 
equally well among occupations, but that different reinforcers are most impor- 
tant for separating the groups. The conclusion from the discriminant analyses 
is that, whereas the unadjusted scales provide limited but useful information, 
Che adjusted scores not only fail to provide any new information but they also 
mav be less interpretable than the unadjusted scores. Some specific recommended 
uses of the ORP scores should be reexamined. Possible explanations for the 
failure of the scaling method to produce valid absolute ORP scores are discussed. 
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Hov Valid are Occupational Retnforcer Pattern Scores? 

A fundamental assumption in much vocational and industrial psycboloRV 
is that a good match of a worker's abilities, interests, and needs with the 
rfquirc»ments and reinforcers of a job promotes satisfaction and perf ortnance* 
The Minnesota Work Adjustment Project represents the most systematic and 
comprehensive attempt to develop both a theory of work adjustment based on 
this assumption (Lofquist ir Dawis , 1969) and a set of methods for measuring 
pf>rson-joh match. The Minnesota Satisf actoriness Scales (Gibson, Weiss, 
Dawis & Lofquist, 1970) and the Minnesota Satisfaction Questionnaire (Weiss, 
Dawis, England & Lofquist, 1967) measure worker satisfac toriness and satis- 
faction. According, to the theory, satisfacrcrii.ess depends upon the match 
between worker abilities, as measured hv the Honeral Aptitude Test Batterv 
(r.S, Deparcnent of Labor, 1970), and job requirements, as measured by the 
Minnesota Job Requirements Questionnaire (Desmond & Weiss, 1973), According 
to the theory, satisfaction depends on the match between worker needs, as 
measured by the Minnesota Importance Questionnaire (Gay, Weiss, Hendel, Dawis 
& Lofquist, 1971), .\nd job rcinforccrs* miiasured hv the Minnesota Job* 
Description Questionnaire (Bor^;en, Weiss, Tinsley, Dawis & Lofquist, 1972; 
Rosen, Hendel. Weiss, Dawis 6 Lofquist, 1972). Thus there is a highly 
structured measureTnent device for each of the kev variables. The major 
objective f/f this theory and its associated measures is *"o help workers choose 
occupations which they will be satisfied, perform well, and persist. 

This report assesses the validity of one of these key measurement devices 
the Minnesota Job Description Questionnaire (MJDQ) ^^rhich is used to measure 
occupational reinforrer patterns (ORPs) • ORP profiles have been published 
for 148 occupations (Rosen et aK , 1972; Borpen et al., 1972), but evidence 
For the validitv of these reinforcer patterns remains linited. The Work 



Adjustment measurement devices are not widely used at present, but it Is 
desirable co have further evidence of their validity should they be adopted 
in .ipplU-d counseling activities. In addition, examination of the construct 
validity of the ORP proflK-s reveaU the hazards of applying psychometric 
scaling techniques to the measurement of Job attributes without first assessing 
their applicability to the specific scaling problem. 

Borsen. Weiss. Tinsley, Dawis and Lofquist (1968) have sutnmarized 
vaUaity evidence for the first 81 occupations for vrtiich reinforcer pat- 
terns were available. Of the 22 scales of the MJDO (21 reinforcers and 
the neutral point^ , all were useful to some extent in distinsuishirr, among 
the 81 .v.'uparions. Although F-tests from one-way ANOVA were significant 
tor everv seal- . ..ccompanv i nr, omee,a-squared statistics ranged from .07 to .37. 
A cluster analvsis based on correlations between the occupational profiles 
resulted in ^ neauin,ful occupational groups. One-way ANOVA for the cluster 
ICmu reinforcer s:ale scores were all significant with omegas-squared between 
.0] ..nd .2',. Bor-en et al. ( PK.S) state that the scale scores of specific 
occupations and clusters i.ake conceptual sense, but they provide few 
examples 'and do na validate the scores against external criteria. More 
rocent analyses hascd on all 148 occupations for which ORP profiles are 
published vieKl essentia- Iv the same results (Rounds, Shubsachs, Dawis C 
l.ot'quist, N(n< 1; Rosen et al . , 1972). 

The only instances in which occupational reinforcer patterns have been 
c6n,pared to cKternal criteria have been studies testing the validity of 
Holland's (197?) occupational tvpologv (Rounds et al., 1978, Note 1; 
roonlos Horr.en. 1974; Oottf redson . 1978b). The results of the Rounds 
.t al. and Toenles and Bor.en studies are confHctinr,, and the Cottfredson 
report concludes that reinforcer pattern scores are not useful for measuring 
absolute levels of reinforcement. 

O 



if there were considerable evidence supporting che propositions of the 
theory of Work Ad.lustment, this could be considered at leaat weak evidence 
for the validity of the ORP profiles. However, evidence for the most relevant 
proposition is conflicting. Several studies (Betz, 1969; Lofqulst i Oawis, 
1469) hav^ found job satisfaction to be correlat.'d with thf degree of correspon- 
dence between a worker's needs and ORP profile, but others (Warren, 1970; 
Taylor, 1971; Willoughby, 1971; V-essey, 1973) found no evidence for such a 
relation. More evidence is needed, therefore, about the validity of some or 
all reinforcer scales for use in either counseling or research on job classi- 
f ication. 

The present rt-port addresses three questions related tr this concern:. 

1, Do the reinforcer scales show converRent validity vlth independent 
measures of the same or similar reinforcers. and divergent validity with 
treasures of unrelated constructs? 

2. How does the method of constructing the reinforc.-r pattern profiles 
affect the validity of pho scores? 

}, Should reinforcer scores be interpreted somewhat differently in 
counseling and research than they now are? 

These questions are answered in the following ways: (a) The procedure 
for deriving' occupational reinforcer patterns is described. As discussed 
further below, the procedure consists of two steps, first developing a set 
of ipsative scores referred to as unadjusted scores and then transforming 
these scores to reflect absolute levels of reinforcement. The latter are 
referred to as adjusted scores and they are the scores published by the Work 
Adjustment Project, (b) Potential problems in interpreting both adjusted 
and unadjusted scores are discussed and illustrated, (c) The properties of 
both kinds of reinforcer scores are compart=>d. Although adjusted and unadjusted 



scares differ, many of their pr^fiertiM «r« «inill«f be^atsBe ef «h« aetliod u«e^ 
to convert unadjusted to adjusted swMres. Knowledge of the ways In «lrtch the 
two scores are related is required because superficial differences ia their 
properties are sometimes assumed to imply that the adjusted scores are more 
valid, (d) The construct validity of eight reinforcer scales is examined in 
decall by correlating them with U.S. Census and Employment Service data about 
occupational characteristics, (e) . Hypotheses concerning differences in the 
substantive interpretation of adjusted and unadjusted scores are tested. 

The Derivation and Interpretation of Occupational Reinforcer Patterns 

Occupational reinforcer scores are obtained for each occupation in 
Identical but separate applications of Gulliksen's (1964; Gulliksen & Tucker. 
1961) procedure for scaling comparative judgment data. Supervisors of, and 
workers In, a particular job are asked to rank 21 reinforcers (shown In Table 
^2) according to how w^ll they describe that job. Rankings of reinforcers are 
obtained from each rater in a paired comparisons procedure using balanced 
incomplete blocks of five items each. The proportions of raters (say 
elementary school teachers) who rate each reinforcer (say compensation) as 
more descriptive of their job than each other reinforcer in turn (security, 
fairness of company policies, etc.) are then calculated. The resulting 21 
proportiom; (.50 being used as the proportion of tiroes a reinforcer would 
have been ranked over itself, had such a comparison been made) are averaged 
for the reinforcer in question and then the average is transformed to a 
normal deviate (z) score. These transformed scores are referred to as the 
un adjusted reinforcer s cores for an 9Ccupation. 

The unadjusted scores are ipsatlve scores, that is. they provide a 
profile of the reinforcers that arc most and least salient within an occu- 
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pat Ion. Because the scores are ysative, iaterocctipatlowal comparisons* ufilttg 
the unadjusted scores should be limited to statements about differences in 
profiles such as "elementary school teachers say that security is a more 
prominent reinforcer than is compensation in their profession, whereas the 
opposite is true for real estate salesmen.'^ The unadjusted ^^res cannot 
be used to make statementa^about which occupation provides higher absolute 
"levels of either compensation or security. To enable comparisons of the 
absolute level of reinforcers to be made, the Minnesota group has devi&ed 
adjusted reinforcer score s by establishing a zero point for each of the 
occupations which is then used to adjust scores. To establish this zero 

point, raters are asked to say w»^ether each reinforcer is present or not 

0 

present In an occupation. The average proportion of the 21 reinforcers 
judsed not to be present in Che occupation is converted to a normal deviate 
score and becomes the "neutral point." Reinforcers with unadjusted scores 
above this point are Judged to be present in the job and those ffelow are 
judged to be absent. To create the adjusted scores, the neutral point for 
each occupation (e.g., z scores of -.869 tox elementary teachers and -.674 f( 
teacher aide) is subtracted from all 21 unadjusted scores wi^tilin that occupa 
tlon. 

Despite the developers' intentions, neither theory nor e/ldence imply 
that this procedure provides absolute scores v^lch are coipfiarable across 
occupations. There is nothing explicitly In common across the assessments \^ 
^y which to create a common scale (Angoff, 1971). For example, raters were 
not asked to compare levels of reinforcement for the same reinforcer in 
different occupations. Keitber were raters asked to rate more than one 
occupation. Note that the only thing that Is changed by adjusting scores 
la the le^ of a profile* The shape of occupational reinforcer pattern 



profiles is not changed at all by the adjustment prooedure» Stated another 
way, the scale Intervals remain exac^^ly the saroe within each occupation; 
only rh6 zero point Is shifted. As Hicks (1970) notes, ipsatlve scores 
cannot be transformed to absolute scores by a simple mathematical trans* 
formation. • " 

An exarapU' using teachers and elementary school aides provides a concrete 
illustration of the failurt; of the adjustment process to create absolute scores 
comparable across occupations. For elementary school teachers, -.869 was 
subtracted from the unadjusted scores of .03 (working conditions). -.75 
(compensiLlon) , and .83 (trv out own ideas) to provide adjusted scores of .90. 
.12, 1.70. For ti'acher aide, the scores for the same three reinforcers were 
adjusted from -.42, and -.04 to 1.21, .26, and .63. Although it is 

plans il>U> that cltwntary teachers have more freedom to try out their own 
Idt-as than do teacher aides (adjusted scores of 1.70 and .63, respectively), 
it is not plausibU- that teachers enjoy poorer absolute working conditions 
and compensat'ion than teacher aides (respectively, .90 and .12 for teachers, . 
hut and .2fy for aides). It is reasonable, however, to conclude from 

the unadjusted scores that r elative to the other r einforcers compensation 
and working conditions are more important in teacher aide jobs than in 
t'lementary teaching Jobs. This is but a single example of the failure of 
adjustment to provide absolute scores: the following sections examine the 
properties of the two kinds of scores more analytically. 

Compariso n of unad j usted and adjusted scores 

It is argued here that the two sets of scorfes provide different types of 
Information, and hence that it is not obvious which set is more useful for 
different purposes. Because this conclusion contradicts the advice of the 

developers, who state that the adjusted scores should always be used (Borgen 

/ 
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et al., 1972:8; Rosen et al.» 1972:8), it is also necessary to examine the 
properties that seem to suggest differences in the validity of the two sets 
of scores and to note important properties of the scores that are easily 
overlooked. 

Correlations among the adjusted and unadjusted scores are shown in 
table 1 for the 148 occupations for which ORP ii.ua are published. The dia- 
gonal contains the correlation between the corresponding adjusted and unad- 
jusf^d scales. The table also shows the correlation of the 21 reinforcer 
scales with the prestige level of the occupations. (The prestige scale Is 
explained further below.) This table reveals some of the differences between 
the two sets of scores: correlations are higher and there are fewer nega- 
tive correlations in the set of adjusted scores than in the unadjusted set. 
The latter point, in particular, has been r.iken (anonymous reviewer, 1978) as 
evidence of the greater validity of the adjusted rcores. But these two 
differences are an artifact of the adjustment pro -.'dure and the two sets 
of correlations themselves correlate .99. By adding the same constant to 
each reinforcer within an occupation (a different constant for each occupa- 
tion), all correlations are necessarily made more positive and the number 
of negative correlations is thereby reduced. Although the number of nega- 
tive correlations is reduced, implausible correlations remain — especially 
for relnforcers that are negatively correlated with prestige. The negative 
correlation of "fairness of company policies" with prestige is implausible. 
The positive but extremely low correlation of '/paid well relative to other 
workers" diverges sharply from other evidence about the association of pay 
.ind prestige (e.g., Reiss et al., 1961). Results to be presented later in 
this paper show that the correlation between the pay «?nd prestige of 
occupations is about .8. 



13 



8 



Insert Table 1 About Here 



Turning now to a ^^ojld type of correlation matrix, we find a different 
pattern (which is unnecessary to present in table form). Correlations among ^ 
Qccajfiations (i.e.. among the row vectors rather than the column vectors of a 
matrix where each of the 148 rows consists of the 21 scale scores for a 
different occupation) are exactly the same for the adjusted and unadjusted 
scores. This is because the profile shapes are not changed in the adjust- 
ment process. Elevation of the profiles changes, but elevation does not 
affect the<ie correlations in any way. 

Analyses based on such correlations among occupational profiles will 
U-ad to the same results whether adjusted or unadjusted scores are used. 
For example, the cluster analysis reported by Rosen et al. (1972) would be 
the same had they used unadjusted instead of adjusted scores. This is not 
true for analyses uslnR correlations among items. Although the shift 
upwards in the Item corr.-lations is systematic when the adjusted scores are 
created, the infomiatlon reflected in those correlations differs from that 
reflected In the unadjusted scores. The following paragraphs describe what 
that shift in information entails. 

The first thing to note Is that the correlation between the prestige 
of occupations and their neutral points is .50 (see Table 1). (For ease of 
interpretation, the sign of the value of the neutral point has been reversed.) 
Thus when scores are adjusted by adding the (sign-reversed) neutral point, 
more is added to every reinforcer within an occupation when the occupation 
is high prestige than when it is low prestige. The result is that if the 
resulting scores are treated as at least interval level measures, all 
occupations beetle less different in those reinforcers that characterize 



14 



9 



low-prestige jobs and more different In those reinforcers that characterize 

high-prestige Jobs. 

Table 2 (with data taken from Gottfredson, 1978b) illustrates this 
phenomenon. The first two columns show the correlations of the 21 unadjusted 
and adjusted scales with prestige level of the occupations. The reinforcers 
are listed from high to low according to the correlation of the unadjusted 
scales with prestige. The third and fourth columns show the variance of 
the scale scores. The last two columns are omegas- squared from an analysis 
of the association between the 21 individual scales and a 17-category 

occupational classification. (The 17-category classification was constructed 
from^olland's [1973] 6 categories of work and 3 broad prestige levels, one 
of the possible 18 cells being vacant.) 

The third and fourth columns show that the variance in relnforcer 
scores decreases for some reinforcers but increases for others ,when scores 
are adjusted. Variance Increases for reinforcers prominent in high-prestige 
Jobs (those at the top of the list) and decreases for reinforcers prominent 
In low-prestige jobs (those listed at the bottom). The Increase in variance 
(column 4 minus column 3) is correlated .89 with the reinforcer's relation 
to prestlRC (column 1). 



Insert Table 2 About Here 



The implications of these changes in variance can be traced with the 
aid of columns 5 and 6, which show the association between each relnforcer 
and the 17-category job classification. Looking first at the reinforcers 
top of the list, it is apparent that omegas-squared are larger for 
the adjusted scores. Further down the list, however, the omegas-squared are 
frequently higher for the unadjusted scores. Omegas-squared tend to be highest 
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for adjusted scores highly correlated with prestige; they tend to be highest 
for unadjus ted scores at the two ends of the continuum— for reinforctrs highly 
correlated positively or negatively with prestige. Although Table 2 shows 
analyses for only a single scale at a time, it suggests that both sets of 
scores distinguish lob categories equally well (e.g., average omegas-squared 
are the ^amt for both) but that they differ in wh ich particular reinforcers 
make the best distinctions. 

The threats to validity discussed above can be summarized as follows, 
(a) It Is not clvar logical Iv chat the adjustment procedure should produce 
absoluto scon-s comparable across occupations. Profile elevation is changed, 
but profile shape remains unaltered, (b) Some of the scores — particularly 
for scales ne-atively correlated with prestige (and the neutral point)-~do^^ 
not seem plausible, (c) The scale developers state that the unadjusted 
scales are n.it valid for makinr. interoccupational comparisuns of absolute 
reinforcor Uvels hut that the adjusted scales are valid for that purpose. 
Hemenstrat icns of validity iiave rested primarily on the ability of the scales 
to dist ini',uish amnns', cnfupat ions . However, Table 2 indicates that both the 
unadjusted ami the adjusted score-s distinguish eaually well among occupa- 
tions, (d) The reinforcers that distinguish best among occupations differ 
depending', on whether adjusted or unadjusted scores are used. This differ- 
ence, as well as others discussed earlier, is an artifact of the adjustment 
process and Implies nothing about the relative validity of the two sets of 
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Strategy for Fjtnniln i nj^ .tJl^, Udi tv _of Re I nf orcer Scales 

This paper further examines the construct validity of Individual adjusted 
rc Inf orcer scales hv correlating them with external measures of the same 
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variables. Eleven predictions are made that should be true if the adjusted 
scales are valid for measuring absolute levels of reinforcement. In addi- 
tion, discriminant analyses are performed to examine the efficiency of 
adjusted and unadjusted scores for discriminating among occupations. 

Validity of individual adjusted reinforcer scales . Adjusted scores 
are intended to measure the absolute level of a reinforcer within an 
occupation and Ideally should correlate highly with other measures of 
the same reinforcer. The U.S. Bureau of the Census (1973), the Dictionary 
of Occupational Titles (U.S. Department of Labor, 1965), and Temme (1975) 
provide estimates of job characcerlstlcs , some of which are similar to 
relnforcers measured by the Woj^ Ad jus tment Project. The 21 job charac- 
teristics examined here include pay, years of education, prestige, self- 
direction, 12 Dictionary of Occupational Titles (DOT) temperaments, and 5 
DOT Interests. A total of eleven predictions Is made for eight of the 
relnforcers; these are listed In Table 3. 



Insert Table 3 About Here 



Judgments about the validity of the adjusted reinforcer scales will 
be based on three considerations. First, an adjusted reinforcer scale 
should correlate higher with the job characteristic It Is supposed to re- 
flect (Che validation criterion) than with any other criterion. Second, 
this correlation (the validity coefficient) should be appreciably higher 
than the average correlation of that reinforcer with other job character- 
istics- And third, because adjusted scores are asserted to be more valid 
for measuring absolute levels of reinforcement than are unadjusted scores, 
the validation criteria should be more strongly related to the former than 
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CO the Ucter scales. The greater validity of an adjusted reinforcer scale 
should be shown by a greater gap between Its correlation with the validation 
criterion and its average correlation with the other criteria than Is the 
case for the unadjusted scores. The gaps between validity coefficients 
and average correlations rather than the validity coefficients themselves 
8ie compared because It Is possible that all correlations are higher In one 
set than In the other. 

Differentiation among occupational groups . Discriminant analyst are 
performed with both adjusted and unadjusted scores to determine how each 
set of scales differentiates among occupational clusters identified by 
Rosen et al. (1972) using ORP profile data. On the basis of the evidence 
presented in Tables 1 and 2 , five predictions are made. (a) Both sets of 
scores will account for the same amount of between-group variance. (b) Both 
sets will place tVie clusters In similar positions in multidimensional space, 
(c) Different discriminating variables will be most Important In the two 
sets. (d) One major dimension will be highly correlated with prestige, 
(e") The prestige-related dimension will be defined by relnforcers highly 
descriptive of hlgh-prestlge jobs for adjusted scores, but by relnforcers 
descriptive of both high- and low-prestige jobs for unadjusted scores. 
,Piit another way, the unadjusted prestige-related discriminant function will 
be bipolar whereas the comparable adjusted discriminant function will be 
unipolar. In short, both analyses will produce Intelligible results, but 
they will describe occupational clusters differently. If the results for 
adjusted scores can be predicted from a knowledge of the correlations among 
the unadjusted scores, the neutral (zero) point, and (to make predictions even 
easier) prestige, th«n It is likely that such results tell us nothing new. 
They may even distort what we might have learned from tne unadjusted data. 

is ' 
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Method ^ 
Published Census data on median Income and years of education as well 
as scores for prestige, self-direction, and im interests and tempet^n«nts 
were assigned to the 148 ORP occupations. The DOT data were available flMT 
DOT occupational titles; the remaining data were available for the more 
highly aggregated detailed census titles. The DOT codes had been assigned 
previously to the 148 occupations by Dawis and. Lofqulst (1974). The Census 
bureau's Alphabetical Index of Industries and Occupations (1971) was used 
to assign the 148 occupations to detailed census titles, the 148 titles 
being distributed to 120 of the possible 428 categories. Two research 
assistants independently assigned the occupations to census categories 
and I resolved the 14 cases where the assignments differed. The job char- 
acteristics and their sources are described briefly below. 
Job Characteristics Data 

Census data on pay and education . Median Income and median years of 
education were obtained from Table 1 of the Census Bureau's 1970 subject 
report on occupational characteristics (U.S. Bureau of the Census, 1973). 
These data are for the detailed occupational titles reported by males in 
the experienced civilian labor force. 

DlctLonarv of Occupational Titles (DOT) temperamen ts and Interests. 
The U.S. Employment Service has developed the DOT (U.S. Department of 
Labor, 1965) over the last four decades for the classification and place- 
ment of job seekers. The DOT characterises over 20,000 Job titles according 
to work activities, training requirements, and worker trait requirements. 
The DOT scores are estimates made by occupational analysts usually on the 
basis of observations of one or more jobs for each occupation (Committee 
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on Occupational Classification and Analysis^ In press). Only the worker 
trait requirements --Interests and tetnperan^nts required by the job*-are 
used here. The 12 temperament Items refer to different types of occupational 
situations to which workers must adjust, such an working alone or doing re- 
petitive work. The 5 Interest items are bipolar Items estimating the pre- 
ferences for certain types of work activities or work experiences required 
by the job, such as preferences for activities of a social rather than a 
nonsoclal nature. Descriptions of the items and how they were coded In this 
study are provided In Table 4. Temperament and Interest score, were assigned 
to the 148 occupations on the basis of DOT codes. Dawls and Lofqulst (1974) 
provided the data for 105 of the occupations. Data for the other 43 were 
obtained item the IX)T - 

Insert Table 4 About Here 

Self-Direction . Sei f-direction Is an Index of the ability of workers 
In a specific job to determine how they will spend their time on that job* 
T e measure used here was originally developed by Kohn (1969) from ratings 
of closeness of supervision, routinlzation of work, and substantive com- 
plexity. Scores were assigned to the 148 occupations on the basis of de- 
tailed census titles. Self -direction scores for the detailed census titles 
have been estin^ted by Tenrnie (1975)* 

Prestige. Occupational status or prestige has been the major dlnH5nslon 
along which occupations have been classified In sociology because of that 
disciplined traditional emphasis on understanding the sources and conse* 
quences of socioeconomic stratification and mobility. Several highly corre- 
lated scales of occupational prestige or sc^loeconomlc status (Duncan, 1961; 
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Taaae. 1975; Treitaan, 1977) have been developed for research on occupa- 
tional attainment. These scales are all tied to ratings by the general 
public of the general desirability of particular occupations, and the scales 
can be considered measures of the general level of rewards provided by an 
occupation. The scale used here and its derivation are described in detail 
by Temine (1975) . 

nrcupational Reinfor cer Patterns (ORP) data . These data have already 
been described, but the treatment of the neutral point, reinforcer item 
names, and ORP occupational clusters should be clarified. 

For ease of interpretation, the sign of the neutral point has been 
reversed. The neutral point (before the sign is reversed) reflects judgments 
about the number of reinforcers that are not present on the job and it ranges 
from -1.10 to -.26. This negative value is subtracted from the unadjusted 
reinforcer scores to produce adjusted scores. This subtraction is in effect 
the same as adding a measure of the average rater's estimate of the number 
of reinforcers present in that occupation to the unadjusted scores. 

Publications by the Work Adjustment Project Investigators use short 
descriptors that do not always convey a clear conception of the full items. 
To avoid such ambiguity In this paper, all reinforcer scales are referred to 
by a shortened version of the items themselves. These versions of the items 
are 1 Isted In Table 2. Further abbreviation is necessary In tables such ar 
Table 1. The labels used in Work Adjustment Project publications which 
cotrespond to the list in Table 2 are, respectively, in descending order: 
responstbillty. creativity, abilitv utilization, achievement, autonomy, 
social status, advancement, authority, social service, variety, recognition, 
compensation, independence, security, working conditions, activity, moral 
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values, co-workers, supervision— technical, supervision— hiraan relations, 
and company policies and practices. 

The discriminant analyses use clusters derived by Rosen et al. (1972) 
using correlations araonp ORP profiles as the measure of similarity. For 
reasons mentioned earlier, the clusters derived would be exa^rtly the same 
whether unadjusted or adjusted scores were used. Although they Identified 
12 clusters, some clusters have been combined with related clusters because 
thev are verv small. Clusters I (N-16 occupations) and II (N-5) were com- 
bined; Clusters V (N-'H) and VT (N»2) were combined; Clusters VII (N-7) 
and VIII {N=18) were combined; and Clusters XI (No9) and XII (N«9) were 
combined. These particular pairs of clusters were combined on the basis 
of the correlation.s among the profiles in those clusters (Rosen et al., 
1972). (Results are much the same using 12 groups, and Interested readers 
are Invited to write for a copy.) 
Llmljcations 

The lob characteristics being used here to assess the validity of 
the adjusted occupational reinforcer patterns data are in some cases of 
questionable validity themselves. The validity of the prestige scale Is 
well studied (Temme, 1975; Trelman, 1977; Relss, 1961), and the pay and 
education data are the best available. The validity of the DOT Interests 
and temperaments is probably the most questionable. In addition, the pay 
and education data assigned to the U8 occupations on the basis of detailed 
census titles do not refer strictly only to those 148 titles because the 
census titles are sometimes more inclusive. This unfortunate but necessary 
use of data organized according to different occupational classifications 
may be expected to attenuate validity estimates somewhat. Where possible, 
judgments concerning the validity of the criterion scales are provided in the 
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Results 

Correlations of Individual Relnforcer Scales with Validation Criteria 

Correlations of the 21 reinforcers with 20 Job characteristics are 
shown In Table 5 for adjusted scores and in Table 7 for unadjusted scores. 
Those correlations about which predictions were made in Table 3 are 
underlined. To help Judge the size of these correlations, the last 

column provides average correlations for the relevant rows. This average 
is calculated from the absolute values of all correlations in ck row except 
those In bold-face type. (The averages are retransf ormed z scores as sug- 
gested by McNemar, 1969:158). 

Insert Table 5 About Here 

-p. «... ...^ * 

Correlations with adjusted scores are discussed first because the pre- 
dictions apply only to this set of scores. Then correlations with the un- 
adjusted scores are discussed for purposes of comparison. A total of 11 
predictions were mnde for G of the 21 reinforcers. The last one of the 11 
predictions cannot be discussed because all 148 occupations scored the same 
on the criterion variable, Temperarent 6. Temperament 6 (working alone 
qnd in physical Isolation) apparently applies only to extrec« cases of 
working alone^^ The remaining 10 correlations for which predictions were 
made are all in the expected direction, but they vary considerably In slste. 
In 3 of the 10 cases (predictions 1, 3, and 8), the relnforcer correlated 
more highly with the validation criterion than with the other' Job charac- 
terlstlcs; In four cases (predictions 2, 6, 9, and 10), the correlations 
with the validation criteria are lower than they are with other Job charac* 
terlstlcs. Looking now at columns rather than rows of correlations for the 
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three best vo» cases, in only one «sse (predictlm 8) 4ees the validstien 

criterion clearly correlate hlghpeat with the correct reinfercer. Thus, the 
validity ooeff icients provide unambiguous evidence for the validity of only 
one of the seven relnforcers for which validation criteria were available. 
More detailed results for the predictions are discussed below. 

Insert Table 6 About Here ^ 

Predictions 1. 2. 3. and 4 . "Making decisions on own" correlated .64 
with self-direction but only -.37 with Temperament 3 (working under specific 
Instructions). Temperament 3 may be a less valid ireasure than Is the rein- 
forcer "making decisions on own" because the relnforcer correlates In expec- 
ted ways with other criteria. For example, that relnforcer correlates nega- 
tively with Temperament 2 (-.54, repetitive work) and positively with Tem- 
perament 4 (.52, planning and directing activities) and Interest 3/8 (.61, 
preferences^ for creative rather than routine activities) as well as with 
job characteristics such as pay (.50), education (.56), and prestige (.60) 
which are higher in decision-making jobs. "Try out own ideas" correlates 
highly with Us two validation criteria — self-direction (.59) and Interest 
3/8 (.66, preference for creative rather than r«nitine activities). 

However, "own decisions" and "own ideas" are not distinct relnforcers. 
Table I shows that they are themselves correlated .83, and the correla- 
tions shown In Table 5 for these two relnforcers are essentially the same. 
It Is not clear that even the next three relnforcers ("use individual abil-. 
Itles," "feeling of accootplishn^nt ," and "plan work with little supervision") 
are distinct from Che first two. Thus, although the evidence supports, the 
validity of the first two relnforcers, they do not appear to measure dif- 
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Predictions 5 and 6 . ''Have position of 'sonebody' In the community" 
correlates .51 with prestige but only -.32 with Interest 5/0 (preference 
for tangible, productive satisfaction, rather than the esteem of others). 
Interest 5/0 may be a poor erasure of status because Table 6 shows that It 
correlated cmly -*44 with prestige. The correlations In Table 6 suggeat 
that Int^mc^^5/0 prlniarlly n^asures working with people versus things. 

r 

Although ''having a position of ^somebwly' In the conmainlty" corre- 
lates .51 with prestige, It correlates almost as well with pay, education, 
and self*-dltv ..--although presumably these all contribute to being 
'•somebody*' In the conmiunlty. Finally, and most Importantly, five other 
re In forcers correlate more highly with prestige than does the relnforcer 
developed to measure It. 

Prediction 7 > The same coiranent can be made about *'telTlng other 
workers what to do" as for "having a position of 'somebody* In the ccmmiunlty. 
That Is, although the relnforcer correlates highest with the valldlation 
criterion rather than with some other Job characteristic, five other rein- 
forcers correlate more highly with that criterion. 

Prediction 8 . "Do things for others" appears to be the most valid of 
the eight rclnforcers examined. The correlation of -.61 with Interest 4/9 
(preference for nonsoclal rather than social activities) Is higher than all 
the other correlations in the relevant row and column. In addition, the 
other high correlations are with job characteristics associated with scK^ial 
service Jobs; education (.44), self-direction (.36), Temperan^nt 5 (.62, 
dealing with people), Temperament Y (*.43, dealing with precis^ limits), 
Interest 1/6 (.47, preference for dealing with people rather than things), 
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and Interest 5/0 (-^2. preference for actlvitleTYesultlng In tangible. ' 
productive satisfaction rather than, eateets) • 

Prediction 9 . "Work Is different every day" correlates only .06 • 
with Temperament I (variety of duties characterized by frequent change). 
This re in forcer does correlate -.43 with a related characteristic, Tempera- 
ment 2 (repetitive op^tions carried out according to set procedures). 
These two correlations suggest that Temperament 1 may be a ^ov measure 
and cannot be used to judge the validity of the reinforcer. ^ 

Prediction 10 . The reinforcer "paid well relative to other workers" 
is clearly not valid. The correlation of this reinforcer with pay is 
only .28. Correlations are almost as high with Temperament 5 (-.25, 
dealing with people). Interest 1/6 (-.24, dealing with people rattfei; than . 
things). Interest 4/9 (.34, preference for nonsocial rather than social 
activities), and Interest 5/0 (-.26, activities resulting in tangible, pro- 
ductive satisfaction rather than esteem). The reinforcer is negatively 
correlated with educatTon and self-directlbn, two characteristics which 
are highly related 'to pay (see Table 6). In addition, eight other rein- 
forcers correlate more highly with pay. Despite any weaknesses the mea- 
sure of pay may have, the oVerall patterns of correlation suggest that the 
variable pay Is clearly more valid than is the reinforcer "paid well."- 
If anything, this reinforcer appears to measure something like "not dealing 
with people." 

If the reinforcer "paid well" is reinterpreted, these correlations make 
more sense. One might suppose that respondents say they are paid well rela- 
tive to other workers when they are paid well relative to other workers of 
their own kind (e.g. of similar age or education). This hypothesis was 
tested by calculating the partial correlation of "paid well" with pay 
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holding education constant. It was assumed that one would feel well paid 
If one earned more than workers with an equivalent education. The partial 
correlation Is .47, considerably higher thai the original .28. This Is 
understandable jjecause people In social service Jobs (which correlated 
negatively with "paid well") are paid less than other workers with equiva- 
lent education (see Gottfredson, 1978a). Even granted this relnterpretatlon, 
however, other relnforcers such as "plan work with little supervision" still 
correlate more highly with pay. Also, this relnterpretatlon means that the 
relnforcet does not n»asure what It was Intended to--the absolute level of 
relnforcenient relative to other occupations. 

Correlations with other relnforcers . Although several relnforcers 
show evidence of validity, "paid well" and several relnforcers below It In 
Table 5 show no such evidence of validity. "Having steady t iployment" is 
not correlated with any of the variables It should be correlated wlth--pay, 
education, or prestige. Although one could argue that bosses train and back 
up their men better In low-paying, low-education, and low-prestige jobs, 
tihia Is contrary to expectation. It is also unlikely that companies admin- 
ister their policies more fairly with lower-level workers. The foi-egoing 
four relnforcers appear to be highest In repetitive work (Temperament 2) 
allowing little self -direction (self-dlrectloi) or Independent judgment 
(Temperament 3) or planning and^dlrec tlon of activities (the absence of 
Temperament 4). These criticisms are consistent with the earlier discussion 
about Implausible correlations with prestige. 

" The problem appears to stem from the Ipsatlve nature of the rel.tjforcer 
scores. The relnforcers low on the list are moat descriptive of low-leVel 
Jobs (I.e. they are negatively related to prestige). W>»at Is distinctive 
within these occupations may not characterize well their relative standing 



22 



amo njg occupations. 

Validity coefficients of unadlusted scores . The adjusted scores are 
supposed to be more valid for measuriog absolute levels of relnforcers 
than are the unadjusted scores. Table 7 for unadjusted scores suggests 
that this Is not the case. Although correlations differ between Tables 
5 and 7, the patterns of correlations In Table 7 suggest the same conclusions 
as do those In Table 5 about the validity or invalidity of the individual 
reinforcers. Overall, the correlations for the unadjusted scores appear to 
be as useful as those for the adjusted scores. This conclusion is reached 
by calculating deviations of the underlined validity coefficients from the 
average row correlations shown in the last columns of Tables 5 and 7 (using 
z transformations of those correlations). On the average, the validity 
coefficients exceed the average correlations by similar amounts: .24 for 
the adjusted and .20 for the unadjusted scores. In this sense both adjusted 
and unadjusted scores are almost equally valid (or not valid) for measuring 
absolute levels, thus calling into question the success of the adjustment 
procfdurc fi'ir obtaining absolute scores. 

Insert Table 7 About Here 

Plscrlmlnan-t Analyses 

The 21 reinforcers were used as the discriminating variables in a 
discriminant analysis of the eight occupational clusters described earlier. 
Analyses were performed separately for adjusted and unadjusted scores. In 
both analyses the first two functions accounted for almost 80% of the 
between-group variance, so the following discussion will be limited to 
those two functions. All hypotheses were confirmed. 

The discriminant analyses show that both sets of scores are equally 
useful for distinguishing among occupational clusters. The solutions are 
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highly similar In several ways. The first discriminant function accounts 
for 59% and 53% of the between-group variance, respectively, for the ad- 
Justed and unadjusted scores. The second discriminant function accounts 
for 21% and 24% of the between-group variance. Also, the centroids are 
nearly identical across the two analyses on both the first- and second 
functions. In other words, when the eight groups are plotted according to 
centroids on the first two discriminant functions, their positions in space 
are almost Identical for the adjusted and unadjusted scores (see Figures 
1 and 2). 

At the same time, differences appear in the coeffic ents of the dis- 
criminant functions, and they Illuminate the changes in substantive inter- 
pretation that can occur when scores are adjusted. The differences are 
best illustrated by referring to Figures 1 and 2. Figure 1 displays the 
results for the discriminant analysis using adjusted scores. Figure 2 
provides the same summary for unadjusted scores. The eight clusters are 
plotted according to their mean scores (centroids) on the first two dlscrlm- 



i^nt functions. The vectors of the 21 relnforcers in this 2-8pace are 
also shown In order to provide a quick Impression of how each reinforcer 
relates to the discriminant functions and separates the occupational clus 
ter3. Although prestige and the neutral point were not used in the calcu 
latlons of the discriminant functions, a vector for each is Included to 
show how they are related to the distinctions among clusters. The direc- 



tion of a vector (cosine-®-) indicates how that variabia Is /correlated with 
the two functions. The length of a vector Indicates how useful that rein- 
forcer is for separating the groups (I.e., It is the ratio of between- to 
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wlchla-grc^p variance f^r that variable). See Overall and fClect (1972) 
for a description of this n^thod of presenting results* 

Insert Figures 1 and 2 AbcHit Here 

Mean prestige of the eight groups is correlated .9 with their cen- 
troids on function I. This is true for both adjusted and unadjusted scores 
and is Illustrated by the vectors for prestige in Figures 1 and 2. The 
first and nost Important function Is the prestlge-related din^nslon. The 
(reflected) neutral point Is also c«Welated .9 with the first function. 

Although the groups are positioned almost identically in space In the 
two figi^res, the patterns of vectors are different. These vectors slmnly 
illustrate in a more dramatic way the differences between the standardized 
discriminant function coefficients (not shown here) for the two^lpts of 
data. One difference between Figures 1 and 2 is that more vectors are 
positive ali>n^' the first function for the adjusted scores than for the 
unadjusted* This occurs because all relnforcers become more positively 
correlated \Jith prestige (which as noted above Is correlated .9 with the 
first function) when scores are adjusted. A second difference Is that the 
relative importance of different relnforcers for separating the groups 
shifts dramiUically. For example, the vectors for "try out own Ideas" 
and ''use individual abilities" are much longer In Figura 1 but the vectors 
for "friendly coworkers" and ''do things for people" are much shor^er than 
In Figure 2. 

The relnforcers which best differentiate the groups in Figure I are 
"do things fur people" and the five relnforcers which load heavily on 
Function l--"try out own Ideas," "use Individual abilities," "make decisions 
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on own/' "plan work with little supervision/' and ''feeling o£ acconapllshment . '' 
The latter five are the five relnforcers most positively correlated with 
prestige. In Figure 2 the most useful relnforcers for discriminating among 
the groups are "do things for people," the three relnforcers with the highest 
positive correlations with prestige ("try out own Ideas," "use Individual 

9 

abilities," and "make decisions on own"), and three of the four relnforcers 
with the highest negative correlations with prestige ("friendly coworkers/' 
"bosses train men well," and "fairness of company policies"). As predicted, 
Function 1 is unipolar with adjusted scores and bipolar with unadjusted 
scores . 

The interpretation of Figure 2 for unadjusted scores Is limited but 
fairly straightforward. For example, "friendly coworkers" and "doing things 
tor people" are the most distinctive relnforcers within Cluster 4 whereas 
the prestige-related relnforcers are most distinctive within Cluster 1. 
Because the scores are ipsatlve, what Is distinctive within an occupation 
may not be particularly outstandinj? in absolute terms when all occupations 
^..r Lop.^ ' '.itin' . If 19 rclnl:orrorr. nrc nracticallv nhncnt in a nartictilnr 
occupation, the remaining two are by default distinctive. Turning to the 
vector for the neutral point and prestige level, we see that the number of 
relnforcers present in an occupation (the reflected neutral point) Is strongly 
related to the g eneral d e sirability (prestige) of an occupation^ It is 
also clear that the major dimension discriminating among the occupational 
groups (given these particular 21 relnforcers) is a prestige dimension. We 
can also see which particular relnforcers are most highly related to this 
dimension, meaning that when these relnforcers are present in an occupation 
that occupation generally has many relnforcers. This observation must be 
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tempered with th« knowledge that SOTJe of the reinforcers may not be distinct 
reinforcers at a a (as was illusttated earlier by their extranely high 
intercorrelatloniO . 

Figure 1 and Figure 2 differ, but the important question Is whether or 
not Figure 1 with adjusted scores tells us anything that Figure 2 with unad- 
justed scores dees not. The distinctive features of Figure 1 were predicted 
from Insights with the unadjusted scores, so these distinctive features 
cannot be expected to provide us new or different information. What is 
more likely Is that Figure 1 actually obscures what can be learned because 
two types of Information (unadjusted scores and neutral points) that were 
kept separate in the analysis for Figure 2 were combined in the analysis for 
Figure 1. In short, not only are adjusted scores no more valid than the 
unadjusted for measuring absolute levels of reinforcement, but they may 
even be less interpre table. 

Discussion 

This study is the first to validate occupational reinforcer patterns 
scores against txternal measures of the same or similar reinforcer constructs. 
Seven of the 21 reinforcers were compared to data on occupations from the 
U.S. Bureau of the Census and the U.S. Department of Labor. The nwjor 
^'-'tntlonr. of the studv are a-, follows- (a) Only seven reinforc°rq p^c • 
examined in detail, (b) Some of the criteria against which the reinforcers 
were validated are themselves of uncertain validity, (c) The occupations 
for which the validation criteria are available are not exactly the same as 
those for which occupational reinforcer patterns data are available. Never- 
theless, the results lead to clear conclusions about the validity of the 
adjusted scores for measuring the absolute level of reinforcers. 
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Conclusions About VaXidlty 

Firs*t, reinforcers having substantial positive correlations with 
prestige — such as "tnake decisions on own*' and "try out own ideas"— show 
evidence of construct validity when tested against external criteria, 
although they all largely measure the same thing. Some reinforcers nega- 
tively correlated with prestige— -such as "fairness of company policies" 
and "bosses back up men"— could not be examined in detail for lack of 
validation criteria, but implausible correlations suggest that they are not 
valid for measuring absolute levels of reinforcement • The results show 
that the reinforcer "paid well relative to other workers" is not a valid 
measure of relative pay, 

r 

Second, although the adjusted scores are supposed to be more valid for 
measuring absolute levels than are the unadjusted scores, they are not. 
Validity coefficients show that both sets are equally useful (or not useful) 
for measuring absolute levels of reinforcement. Both sets of scores also 
dlf fi rentiato groups of occupations equally wel] in the discriminant analyses. 

Third, knowledge of the unadjustf?d scores and neutral points is suffi- 
cient to predict the message that a discriminate analysis of adjusted scores 
will Klve us. The analysis of adjusted scores fails to provide any new 
information. 

Re commend ations for Resea rc h and Practice 

Both adjusted and unadjusted scores produce the same profiles. This 
paper did not examine the validity of the scales for constructing occupa- 
tional profiles nor for using them to create an occupational classification. 
But whatever the validity oi the profiles, it is the same for the adjusted 
and unadjusted^^ scores. Only the adjusted scores are designed to measure 
level of the profiles. This paper has shown them to be of questionable 
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validity for measuring absolute levels of reinforcers. Therefore, at least 
those uses that take level of profiles into account must be carefully re^ 
examined* 

Of the uses recommended by Rosen et al. (1972) and Borgen et al. (1972), 
the following deal with profile shape only and are not, therefore, necessarily 
problematic: visually matching profiles of counselee needs with the rein-- 
forcers present in various work environments, identifying occupations with 
similar reinforcer patterns, and listing which reinforcers are most and 
least distinctive within different occupations. All interpretations should 
of course be made with the clear awareness that what is distinctive is not 

Iv plentiful. The computer scoring service for the Minnesota Impor- 

n 

tanre Questionnaire (MIQ) uses D" to match ORP profiles with the needs pro- 

2 

files of counselees (Cay et a I., 1971). D is the sum of squared differences 
between an individual's Minnesota Importance Questionnaire (needs) adjusted 

* 

scale values and the corresponding adjusted scale values for the ORP of a 

2 

specific occupation. Although D is affected by profile elevation, its 
use mav not bo problematic because the MIQ needs scores are adjusted (ele- 
vated) by the same procedure as are the ORP scores. 

Other recommended uses that take profile level into account should be 
re-examined. Ranking scale values for single scales is recommended (Rosen, 
i't j1., 1972:p. 15; Borgen et al., 1972:p. 14) when counselors have Infor- 
mation only about what the counselee' s roost imnortant need Is, If that 
primary need were pay, for example, the counselee would be better served by 

f 

providing census data or other sources of data about pay than by examining 
the invalid "paid well relative to other workers"i» reinforcer scale values. 
A related questionable practice is to interpret mean scale differences 
among occupational groups as absolute differences In reinforconent levels. 
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For exoaple, the factors In Tlnsley and Weiss's (1974) investigation of 

2 

the relnforcer structure of occupations need to be interpreted diffeiently. 
This caution would apply to using reinforcer means to Interpret any 

classification of occupations. 

One final recoinmendation is to investigate why the scaling procedure 
failed to produce absolute scores cc^nparable across occupations* The 
problem may be that Thurstone's law of ccwnparative Judgment was inappro- 
priately applied to this scaling task. Although no mention Is made of 
whether or not necessary assumptions are met (e.g., homogen^ty of variance) 
or anchoring effects are present, the discussion will f<H:us on more basic 
potential problems. The adjusted scores were derived In two steps: (a) an 
Interval scale was created separately for each occupation, and (b) a zero 
(neutral) point was created for each occupation and the scale adjusted by 
the relevant neutral point. These two steps created 148 scales (one for 
each occupation), each with a neutral point Indicating which relnforcers 
are present or absent. The second step was carried out In order to provide 
measures of absolute levels of reinforcement and so allow Interoccupatlonal 
ct>mparlsons of those absolute levels. However, one further step is necessary 
to make such comparisons. One must assume (because It Is not demonstrated) 
Chat the 148 scales actually represent only a slnt^le cotmnon scale, for 
example, that the 148 scores for a particular relnforcer themselves form a 
good scale. To do this one must assume both that the neutral points are 
comparable across all 148 scales and that the scale Intervals are Che same 
across all scales. These assumptions are unjustified until supported by 
evidence about the relation of the 148 scales to each other or to a conmton 
standard. 
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One important differetice b«eweeii the ORP procedure and. that of typical 
Judgment tasks tnay provide another related clue to the failure of the OR' 
method. The occupations are analogous to subjects in a judgment task; tie 
racers are analogous to repeated trials by the subject. The relnforcer 
Items are analogous to stimulus objects, but with one crucial difference. 
Stimulus objects are typically constants (a 10-pound weight is always a 
10-pound wieght). But the stimulus objects in the ORP problem are varia- 
bles. In fact, the goal of the scaling procedure is to assign values to 
those variables. In short, the ORP scaling problem involves a 3-dimensional 
matrix to which a 2-dlmensional method was applied. The shift in emphasis 
may be better illustrated by modifying a more typical scaling problem to 
rosemblf the task of the OP.P analysis. The vegetable-rating task used bv 
Borgen et al. (1968) to explain their adjustment procedure will illustrate 
the additional scalluR demand posed by the ORP task. Imagine 148 subjects 
rankinfi the size of 21 vegetables. But the vegetables being rated are of 
varvlnj; sizes. Subject A is always exposed to a large tomato and a tiny 
rucumber; Subject B always Rets a small tomato and a moderately-sized 
cucumber; and so .on. The task for the investigator is then to take the 148 
sets of rankings and calculate the size of the Individual vegetables to 
which each subject was eotposed, e.g., who got the largest tomato. It is 
not clear that the intervals of the scales (adjusted or unadjusted) have 
anything to do with the intervals that might be obtained if a single subject 
were to rank all the tomatoes. 

In conclusion, readers are reminded that this paper has questioned the 
v.ilfdltv of onlv one assumption about the ORP scores— the assumption that 
inferences about absolute levels of reinfor<\ement can be drawn from adjusted 
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reinforcer scales. Althwgh only weakly validated, the unadjusted scores 
and the neutral points provide interesting and intelligible information 
about occupational reinforcers. This information should be exploited more 
fully, not only to show how reinforc6rs are related but also to better adapt 
the ORP profiles for practical application. Other ways of providing scores 
comparable across occupations should be investigated. Earlier alternatives 
that were abandoned because of their cumbersome nature (see Borgen et al., 
1968;pp. 4-6) should' be reexamined. But the applicability of any potential 
scaling method to the special featu^s of the OR? scaling task must be care- 
fully assessed and veridical comparisons made where possible to validate 
resulting scales. 
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Footnotes 

« 

1. A number of the Interests and temperaments in the K)T estimates are only 
rarely rated as present. The distributions of these variables are des- 
cribed by the Committee on Occupational Classification and Analysis (In 

<» 

press) . 

2. In that paper. Tins Icy and Weiss (1974) describe eight occupational 
clusters in terms of mean factor scores on five factors derived fran the 
21 idjvtsted reiniorcers. Those five factors and the relnforcers which 
load most highly on each are (a) creative responsible achievement ("use 
abilities." "feeling of accomplishment," "try out own Ideas," and "make 
dfcisions .in own"), (b) benevolent supervision ("fairness of company ^ 
policies." "bosses back up men," and "bosses train men well"), (c) service 
("do things for people"), (d) work alone ("work alone"), and (e) steady 
comfortable employment ("have steady employment" and "good working condi- 
tions") . Problems with assuming (as Tinsley and Weiss do) that these 
tactors reflect absolute levels of re inf oircement will be Illustrated 

with the first 2 factors. The first factor is defined primarily by the 
four relnforcers with the highest positive correlations with prestige; 
Che second factor by the three reinforcers with the highest negative 
correlations with prestige. If the factor scores are Interpreted as 
absolute levels, the first factor makes sense but the second does not. 
Clusters III and VI are singled out here because the former has the 
next to the highest Factor I mean score and the lowest Factor II mean 
score; the latter cluster has the Its^est Factor I but the highest 
Factor II mean scores. Cluster III is called creatlve/lndependfent/ 
altruism and includes, for example, case-worker, teacher, architect, 
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newspaper publisher, and Interior designer; Cluster VI Is labelled rou* 
tlne/supervlsed/non-servlce/teanTtfork and Includes « for example , assembler, 
meat cutter, pilot, booId)lnder, cashier, postal clerk, and punch press 
operator. The positions of the two clusters on Factor I are sensible, 
but it Is doubtful that the first group has the least benevolent and 
the second the most benevolent supervision of all occupational clusters. 
What is more probable Is that benevolent supervision is more salient 
in the latter. The titles of the clusters themselves suggest this. 
Cluster III receives little supervision in contrast to Cluster VI. 
Similar objections could be raised with Factor V (steady, comfortable 
employment) . 
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Correlations Awjng (k^cupaclonal Relnforcer Pattern ScilM, Neutral Point,. 
And Prrttlge for tK>th Adjusted and Unadjusted Scsles^ 
(N - 148 Occupations) 
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Table 2 



The Relation of the 21 Reinforcers to Occupational Prestige Level, Relnforcer 
Variances » and On^gas Sqitared with 17 Categories of Work 

(N • 148 Occupations) 



Reinforcers 



Correlation of 
Scores with Prestige 
Unad J us ted Ad j us ted 



Variance 



Unadjusted 
(3) 



Adjusted 
(4) 



Omegas Squared for 17 
categories of Work 
Unadjusted Adjusted 



Make decisions on own 


.55 


.60 


.14 


.25 


.37** 


• 43** 


Try out own ideas 


• 51 


.57 


.17 


,28 


.45** 


.47** 


Use indlvldufll abilities 


• 51 


.58 


. 11 


.21 






Feeling of accomplishment 


.47 


.61 


.06 


• 12 


.36** 




Plan work with little supervision 


• ^3 


.60 


.09 


. 15 






Position of '^somebody" in the 








.13 


.22 


.38** 


community 




.51 


.09 




.18 


.38 


.14 


.18 


.24* 
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. 12 


.13 
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.21 
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.20 


Work alorifcf 
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.14 
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.10 


Have steady employment 
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Good working conditions 
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.24 


Friendly co-workers 
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.03 


.22 
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Busses train their men well 
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-.25 


.09 


.08 


.34** 


.14 
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-.20 


.07 
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.39** 


.14 
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-.56 


-.28 


.09 


.07 
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^ * The 17 categories of work were constructed from Hoi land •s 


(1973) 6 categories 


of work and 3 broad prestige leve 


where one of the possible 18 cells 


v%9 vacant. 
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^ The changes In variance when scores are adjusted (column 4 minus colunrn 3) and the correlations between the unadjusted 
scores and occupational prestige (colimm 1) are correlated .89. 
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Table 3 



Validation Criteria for Eight Reinforcer Scales 
and Predicted Direction of Correlation for Adjusted Scores 

Prediction Direction of 

No. Reinforcer Validation Criterion Correlation 



1 

2 



Make decisions on own Self-direction 



3 
4 



8 



10 



11 



ft 



If f I 



Try out own Ideas 



ft II ir If 



Position of "somebody* 
in the cotmnunity 



Tell others what to do 



Do things for other 
people 



Work is different 
every day 

Paid well relative to 
other workers 

Work alone 



Situations involving doing things only 
under specific instruction, allowing 
little or no room for independent action 
or Judgment (Temperament 3) 

Self-direction 

Situations involving a preference for 
activities of a routine, concrete, or- 
ganized nature vs. those of an abstract 
and creative nature (Interest 3 vs 8)^ 

Prestige 



Situations involving a preference for 
activities resulting in prestige or the 
esteem of others vs. tangible, productive 
satisfaction (Interest 5 vs 0)^ 

Situations involving the direction, 
control and planning of an entire 
activity or the activities of others 
(Temperament 

Situations involving a preference for 
working for people for their presumed 
go6d, as in the social welfare sense, or 
for dealing with people and language in 
social situations vs. activities that 
are nonsocial in nature, and are carried 
4pn in relation to processes, machines 
and techniques (Interest 4 vs 9)^ 

Situations involving a variety of 
duties often characterized by frequent 
change (Temperament 1) 

Median income of males 



Situations involving working alone and 
apart in physical isola ion from others, 
although the activity may be integrated 
with that of others (Temperament 6) 



+ 
+ 



^he first Interest of a pair is scored low; the second of a pair is scored high. 
Note. See text for the definitions and sources of the validation criteria. 
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6. 



Unx Tciiip«r4 



^ltu4tliHi» invntvi.m • variety of 4u(l«« 0ft«i chtrsctarls*^ by frc^utffit cMng*. 

Stt«i4ttiii« IrtvotvtnH rap«rittv« or •hnrt cy<.l» or»«rflcton« carried out flccardlns to nmt pr>K«<hirtp ar ««^tf«iic««. 
Sltu«tlon0 Inv^lvln* Mff^ thing* only im»S#r «p««rtflc Initructlon, clloirinR tlttU or no rogw for ic^p^odofit 
action or tudionrnc In w^rfclrtg out t<* f>roblf«». 

SUuntlim* imrcil^inn thm dtro«rt<^, control, and pUtmlii« of an •ntlrc activity or tha «ctivitioa of otltara* 
Situatlona InvolvtnK the nrceiiity of daaling %^irh p«opla in actual Jot dt^imm boyond givias and rM^aivifSf iiWtrMC- 
tiona . 

JUtuationa litvotvlnH w»>rilnii <i!ona and apart In phyalc«l laoldtioct iwm othart, atfhociih th« acti^ty m^y bm ifit«- 
tc(.«ta<i vttH that <«f ntNera. 

SU»Mll.Hi» lovolvlnR Intluritclntt other pat>plt in tK^ir itplniona. attitudoa, ot Judt^to about idc«i or tbiitAi. 
•Utujtlifi* litwIvlrtK perfMrrnlnn ^daqu«t«»ly under ntrmw whan cimfrontad irith tha critical or cmamiactad or vfian 
taking riaki* 

;U.,ttlort» involving thr ivalu;itton tartlvtng at Ran^raUtatlnna, judR^iita, or d««ialofia) of Intomatioti againat 
censor V or tpdionent^l cilt^rl^. 

JltuMii involving the ev.ilu«tU>n (arrlvloK at Raoarallaatlcma. (ud^i^nti. or daclalrmt) of Infomatlona againat 
fli«n«ur^bta or vvtlf labia criteria* * 
mruatlmii lovclvlnK the Intarpretatlnn nf f^-ellngi. Idaaa or facta In tarsw of paracmat vlavpoint * 
♦.Itii^tlona Ifivolvlng the precise iitrilfiment of aet Umlta, tolarancea, or itandarda^ 



Interaata 



Sltu^ti(H«a lnvolvltt^ ^ prefatance 
f.M i^tivitlea '!e4M»»H with thlnga 
.«nd ob Icrctc . 

Slrttitiluna Involvlm: a pr«»lrrenre fi>f 
jctlvttir* Inwlvlim btjiln<»a» cmitact 
vith pe'^plv. 

;ftL4<tl«^ofi Involvlrtg 1 preference t«it 
«rtlvitlri of ^ ri>i»tlne. ton<rate, or - 

ift'itlHOP invitlvt'U' I rfrief<»m'e 
1 11 virklnn Iff pr -vlr t<»r t»«r!r 
i«refViflMi<i i" tHe iinrlHl 

w«H4re ■fm«r, or in dei^ling vttti 
(foopte aod language in •oclal ilt- 
\iatlona . 

Silu-ttiotTf Invf^lvlot* * prefatem:^ 
foi ACtlelrlea re^uftlnH In prr«- 
titie or tha e»ieai« of other*. 



b* . Sltuatlona litvoWing a praf^ranc* for 
va. * activitlaa coocarwd with p«opl« and 

tha coqiMmication of Idaaa* 

J. Sltoationi Invtvlvif^ a praforanca for 
v«, activitlaa of a aclagtlflc dud titch* 

nlcat natura. 

^ 8/ Situatioot involving a prafaranca for 

activitlaa off an attract and craatlva 
nature . 

iQltuatlora involving a prffaranra for 
activltla* that era nonaoclal In na- 
tura» and are carried on In ratatioo 
to procaaaaa, aaebinaa, $nA tacuni^uaa* 



va . 



O. Slcuationa involving a praf«rance for 

actlvltlea reiuttinji In a tangible, pro- 
duct Iva lafia fact ion. 



* Te»»per»wiinlf uere r*wied «■ I if praient . 0 if ibaent. 

^ fTtrerenta were mdrntl •» I If the flr»t half "f the Iti 
^ if fieither pole of the Irew applied. 



applied. 3 If the aaeond half applied, arnl 
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T«bU S 

. CorreUtlocifl of Atijuaeed Reinforcar 
DOT Tempet^mcntm , DOT lnt«r«itf, antf Oth«r 
(N « I'^S occufMtioni) 
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52t.: Underlining iM.tfc^tc, .-..rret-it (.ms f..r w«,Jch rri-dicttwii wt-r^ mad*- In TflhU 3. 
• Refers to mal. exp.rtpnced civilian labor force. 

" Te».p.r.«»nt f, I, exclwded hrcau.e all 148 occup-tl<m. w«r. coded 0 for thi. t«8pflr.»nt 



Avera.. c,.r.,.H..n of th. r.lnforcor wUH -11 criteria .Kcept for the v-lld.tlo„ crlterU for th-t p.rtlc«Ur r.lnf«rc.r. , A^r.»e. .r. 

milU % tr4n^roEm.iLlonf» ttf tfie ^hsnlntp v/i1«.p« of the correct iont* . 
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Table' 6 



Correlations Among DOT Temperaments, DOT Interests, and other Job Characteristics 

(N " 148 occupations) 
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* Refers to male experienced civilian labor force. 
'5? Temperament 6 Is excluded because all 148 occupations were coded 0 for this temperament. 



table 7 



CorrelatiorMi ot Unadjusted Eeinlorcer Scot** trlth 
^ DOT Tcpiperame'nC* « DOT Interttft*, and Other J<^ Characteristics 

(N - 1^8 oecufMtions) 



g|f>te : Underlining l"^Hi^tr«» c<irr#*l.itl«^iis fnrwhlth prod ict I iwis were m.ide in lAbte i. 
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g^fey* to male experienced civilian labor force. 
^ Temp<^xarRen t 6 is excluded because all 148 occupatlws were coded 0 for this temperament. 

^ Averai^e correlit i twi the relnforcer with .ill criteria errept fr»r the validation criteria for that particjilar reiiiforcer. Averagea vere calcnlated 
with /. t r.Ui^ f -T^i i '^ns <if the .ihsfilnte v;ilMr<; nf tlie correl .1 1 i <^^a . 



45- 



55 



56 



46 . 

V 

Figure Caption 

Figure 1, Centroids of eight occupational clusters on the first and 
second discriminant functions and the vectors of the 21 relnforcers, prestige, 
and the neutral point: Adjusted scores 
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Figure Caption 

Figure 2. Centrolds of eight occupational clusters on the first '^nd 
second discriminant functions and the vectors of the '^l reinforcers, prestige 
and the neutral point: Unadjusted scores 
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